Rational Learning of Mixed Equilibria

This paper investigates the problem of policy learning in multi-agent environments using the stochastic game framework, which we brieey overview. We introduce two properties as desirable for a learning agent when in the presence of other learning agents, namely rationality and convergence. We examine existing reinforcement learning algorithms according to these two properties and notice that they fail to meet both criteria. We then contribute a new learning algorithm, adjusted policy hill-climbing, which achieves both properties and is based on a simple principle: \learn faster when losing, slower when winning." The algorithm is rational and we demonstrate it on several stochastic games, showing it to be capable of converging to mixed equilibria.